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Grant  AFOSR  76-2951 
Interim  Scientific  Report 

Technical  Summary 


This  report  describes  the  research  progress  in  the  five  areas:  image  processing 
and  display,  syntactic  character  ieeognition,  feature  formulation  on  reconnaissance 
imagery  analysis,  seismic  pattern  recongition,  and  statistical  pattern  recognition 
with  finite  sample  size.  Effort  cn  image  processing  has  been  concentrated  on  simple 
but  efficient  operations  that  meet  the  realtime  recognition  needs.  This  includes 
histogram  manipulation , image  sharpening,  compression,  filtering,  line  detection, 
etc.  The  syntactic  character  recognition  compares  favorably  in  performance  with 
the  statistical  recognition  but  requires  much  more  computational  complexity . 

Emphasis  is  made  on  the  extraction  of  both  statistical  and  structural  features.  Qy 
using  a classification  tree,  features  are  formulated  sequentially  in  order  to  arrive 
at  consistent  interpretation  on  each  segment  of  a picture.  Factors  that  limit  the 
present  capability  in  automatic  analysis  and  classification  of  seismic  events  are 
described.  For  the  seismic  records  currently  available,  it  is  felt  that  further 
effort  on  using  the  signal  processing/ feature  extraction  mixed  approach  can 
increase  the  performance  limit.  This  implies  the  use  of  phase  spectrum  information 
and  better  seismic  noise  estimate.  A class  of  nonparametric  method  using  k-nearest- 
neighbor  rule  has  provided  surprisingly  good  performance  under  finite  sample  size. 
However  it  is  cautioned  that  in  general  the  performance  degradation  due  to  finite 
sample  size  is  very  significant.  And  such  effect  should  be  taken  into  account  in 
all  aspects  of  recognition  system  design.  Finally  some  effect  of  the  finite 
sample  size  on  information  and  distance  measures  is  examined. 
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Grant  AFOSR  76-2951 
Interim  Scientific  Report 
(March  1976  to  February  1977) 

I.  Image  Processing  and  Display 

During  this  period  two  new  image  data  sets  are  added  to  our  image  recognition 
research.  One  set  contains  21  reconnaissance  images.  The  other  is  U5C  Image 
Processing  Institute  data  base  with  200  pictures.  The  purposes  of  examining  new 
data  sets  are  to  test  the  feasibility  of  software  algorithms  already  developed 
and  to  explore  new  problem  areas.  Although  the  reconnaissance  imagery  has  less 
quality  than  the  USC  data  base,  it  is  particularly  suitable  for  recognition  study. 

It  is  interesting  to  note  that  the  detection  of  airplane  poses  a somewhat  different 
problem  from  the  detection  of  tank  or  truck. 

Regardless  of  the  final  objective  in  image  study,  image  processing  and  display 
is  almost  always  an  essential  step.  Our  effort  in  this  area  has  been  directed  toward 
developing  simple  but  efficient  operations  that  meet  the  realtime  recognition  needs. 
Figures  1 and  2 show  typical  reconnaissance  images  studied  (the  original  digitized 
image  is  8 bits/pixel).  For  the  target  area  in  Fig.  1,  Fig.  3a  is  an  eight  level 
display  on  the  two-level  Tektronix  1*010  terminal.  Figure  3b  is  the  modified 
gradient  of  the  same  area.  (The  modified  gradient  as  defined  in  Ref.  5 is  different 
from  those  proposed  hy  other  authors.)  Figure  3c  is  the  modified  gradient  operation 
performed  on  the  noise  cleaned  data.  Spatial  filtering  and  skeletonization 
algorithms  have  also  been  developed  for  the  reconnaissance  data.  The  two  sides  of 
a road  in  Fig.  1 can  be  detected  by  line  fitting  process  using  Hough  transform  as 
shown  in  Figs.  4a  and  4b.  Inspite  of  the  equipment  limitations,  every  effort  has 
been  made  to  provide  the  best  display  by  using  line-printer,  X-Y  plotter,  and 
Tektronix  terminal.  Effort  is  also  now  made  to  acquire  an  electrostatic  plotter 
which  provides  instantaneous  multi-level  display  at  our  PEP  11-45  interactive 
recognition  computer  system. 


Aerial  view  of  a tank  moving  on  country  road  (file  #6). 
Two-level  display  with  sliced  levels  lljO  and  170. 

Figure  1 


Aerial  view  of  a.n  airfield  (file  015).  Two-level 
display  with  sliced  levels  135  and  175. 


Figure  2 


Fight-level  display  of 
target  (tank)  area  in 
Fig.l  on  a two-level 
display  terminal. 


Fig.  3a 


Modified  gradient  pic- 
ture of  target  (tank) 
area  in  Fig.  1.  Thres- 
hold is  3^. 

Fig.  3b 


Modified  gradient  picture 
of  target  (tank)  region 
in  Fig.  1 on  noise  cleans- 
ed data.  Threshold  is 


Line  fitting  process  for 
specified  angles  in  Houy;h 
transform  operation  for  the 
road  in  Fig.  1. 


Results  from  Hough  transform 
with  two  sides  of  the  road  in 
Fig.  1 fitted  by  lines. 
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Kighleyinan's  character  sets 
for  syntactic  recognition 
study. 

Fig.  5 
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II.  Syntactic  Character  Recognition 

Figure  5 shows  a typical  set  of  handprinted  characters  studied.  Stochastic 
syntax  analysis  is  made  on  characters  P and  R using  one-dimensional  strings.  Ten 
training  sarples  are  used  for  each  class.  The  recognition  result  with  two  errors 
out  of  100  samples  is  fairly  good  as  comapred  with  the  statistical  recognition 
result . However  there  is  more  complexity  involved  in  parsing  than  in  finding  the 
statistical  distribution.  We  feel  that  statistical  approach  is  more  suitable  for 
this  kind  of  patterns  (characters) . 

III.  Feature  Formulation  on  Reconnaissance  Imagery  Analysis 

In  conjunction  with  a preprocessing  study  of  the  reconnaissance  Imagery  reported 
above ,. extensive  effort  has  been  made  to  formulate  an  efficient  feature  set  which 
contains  both  statistical  and  structural  information  and  meets  the  computational 
requirement . Hiis  is  different  from  the  feature  selection  process  in  classical 
recognition  system  in  which  a small  set  of  good  features  is  chosen  from  a large  set. 

A major  purpose  of  the  study  is  a better  understanding  of  the  imagery  through 
detection  and  locating  of  interesting  object(s)  and  analysis  of  its  background. 

Each  image  has  at  least  one  interesting  object  as  shown  in  Figs.  1 and  2. 

The  class  of  features  considered  belongs  to  local  features  (pp.  6l-85,  Ref.  ll) 
as  they  are  much  less  complex  to  extract  than  the  global  features . The  feature 
formulation  is  guided  by  a diverse  source  of  knowledge . The  statistical  information 
includes  the  statistical  parameters  such  as  skewness  of  the  histogram,  the 
resolution-dependent  entropy  measures,  and  the  textural  measures  based  on  a co- 
occurrence matrix.  The  structural  information  includes  an  average  modified  gradient 
measure,  and  the  linear  and  symmetry  characteristics  detected  by  simple  algorithms. 
Each  of  the  features  stated  above  is  simple  to  extract  but  inadequate  to  be  used 
alone  to  provide  good  understanding  of  ^he  imagery.  A classification  tree  is 
employed.  By  starting  with  simple  features  such  as  skewness,  modified  gradient, 
and  entropy,  the  sequential  classification  process  determines  whether  consistent 
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interpretation  can  be  derived  from  existing  features,  what  features  are  to  be 
taken  at  next  step,  whether  additional  preprocessing  is  needed  in  certain  ambiguous 
region,  etc.  Detailed  computer  results  will  be  included  in  a master's  thesis 
(expected  in  May  1977)  supported  by  the  Grant.  Although  human  interaction  is  needed 
at  the  present  study,  we  feel  that  this  is  a feasible  step  toward  a fully  automated 
reconnaissance  imagery  analysis  system. 

Preliminary  recognition  results  show  that  all  tanks,  cars,  trucks  can  be 
detected  correctly.  However,  the  shadows  of  the  airplanes  present  a different 
problem.  The  graylevels  of  airplanes  are  very  close  to  their  background.  The 
shadow  is  usually  recognized  as  the  airplane  itself.  The  information  on  the 
direction  of  light  illumination  and  the  distance  between  the  object  and  the 
shadow  should  be  used  to  remove  such  ambiguity  in  object  detection.  The  (phase) 
angular  information  from  the  modified  gradient  has  also  been  considered  for  this 
purpose . 

IV.  Seismic  Pattern  Recognition 

This  is  an  area  that  signal  processing  techniques  are  much  needed  to  extract 
effective  features.  The  autocovariance  features  and  the  power  cepstrum  with 
alpha-minus-C  energy  estimate  are  both  effective  features  (Ref.  3).  However, 
the  89.32!?  correct  recognition  which  is  the  best  available  indicates  the  performance 
saturation  or  limitation.  Such  limitation  could  be  caused  by:  limited  size  in 

total  seismic  records,  great  variations  among  the  seismic  records  available, 
inadequate  noise  statistics  because  the  record  is  short  and  noise-only  duration 
is  very  small,  etc.  Only  the  bodyvave  information  is  utilized  in  the  study  and 
there  is  no  way  to  separate  the  surface  wave  from  the  bodywave  again  due  to  the 
fact  that  the  record  is  very  short.  However,  to  work  with  the  data  available, 
more  sophisticated  signal  processing  techniques  must  be  used  to  extract  additional 
informations  currently  unavailable.  For  example  the  phase  spectrum  information 
can  be  used  to  generate  new  features.  And  the  structural  decomposition  should  be 
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performed  with  perhaps  the  synthesized  seismic  record  to  derive  features  from  the 
surfacevaves.  Thus  by  increasing  the  variety  of  features  that  contain  statistical 
and  structural  informations,  we  believe  the  performance  limit  can  be  moved  up 
considerably.  It  is  doubtful  that  ary  single  kind  of  feature  can  provide  adequate 
performance.  Again  we  must  emphasize  on  the  use  of  the  statistical-structural 
mixed  model  for  such  complex  pattern  (Ref.  10). 

V.  Statistical  Pattern  Recognition  with  Finite  Sample  Size 

In  Ref.  6 we  have  shown  that  the  lower  bounds  of  the  probability  of  correct 
decision  for  a class  of  sample-based  classification  procedures  losing  the  k-nearest- 
neighbor  rule  can  be  very  close  to  that  obtained  with  the  Bayes  linear  discriminant 
analysis  based  on  the  assumption  of  two  multivariate  Gaussian  densities  with 
different  mean  vectors  but  equal  covariance  matrices.  The  nonparametric  method 
obviously  is  good  for  the  finite  sample  size  under  the  assumed  conditions.  However 
once  the  conditions  are  changed,  the  performance  can  be  changed  considerably.  It 
is  also  noted  that  a recent  paper  by  8.  Pr.udys  which  appears  in  the  Third 
International  Joint  Conference  on  Pattern  Recognition  has  given  a much  more 
pessimistic  performance  under  finite  sample  size.  Thus  the  classification 
performance  under  finite  sample  size  constraint  is  largely  an  unresolved  problem 
and  remains  to  be  further  examined. 

For  the  information  and  distance  measures,  continued  study  has  been  made 

on  the  finite  sample  size  effect.  Consider  cne-dimensional  Gaussian  densities 
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with  zero  means  and  variances  end  ct^  vhi.’i  are  equal.  The  Bhattacharyya 
distance  ii  given  by 
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which  has  the  F-distribution,  with  (II  , ITp)  degree  of  freedom,  given  by 
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The  lower  bound  is  generally  a negative  number.  This  means  that  E(B  - B)  < 0, 
and  that  the  Ehattachaxyya  distance  evaluated  by  using  finite  number  of  samples 
can  lead  to  somewhat  pessimistic  estimate  of  the  error  probability.  Although  this 
conclusion  is  different  from  that  of  divergence,  it  demonstrates  that  the  finite 
sample  size  effect  is  quite  unpredictable. 
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